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CN Abstract 

f^*) Our concern is the computation of optimal shapes in problems involving (—A) 1 ' 2 . We 

focus on the energy J(f2) associated to the solution of the basic Dirichlet problem 
(— A) 1 / 2 ^^ = 1 in O, u = in fl c . We show that regular minimizers f2 of this energy 
D under a volume constraint are disks. Our proof goes through the explicit computation of 

Ph the shape derivative (that seems to be completely new in the fractional context), and a 

^vq refined adaptation of the moving plane method. 
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1 Introduction 



This article is concerned with shape optimization problems involving the fractional laplacian. 
The typical example we have in mind comes from the following system: 



(-A) 1/2 u=lonO, , . 

T—{ K ' (1.1) 

j>. u = on Q c , 

° 

set for a bounded open set 0, of M . We wish to find the minimizers of the associated energy 

On 

J(n):= inf (- /(-A) 1 / 2 !;, v\ - [ v) . (1.2) 

f|nc=0 

(N 

among open sets Q with prescribed measure (and a smoothness assumption). Beyond this 
specific example, we wish to develop mathematical tools for shape optimization in the context 
of fractional operators. 

Our original motivation comes from a drag reduction problem in microfluidics. Recent 
experiments, carried on liquids in microchannels, have suggested that drag is substantially 
lowered when the wall of the channel is water-repellent and rough [16] . The idea is that 
the liquid sticks to the bumps of the roughness, but may slip over its humps, allowing for less 
friction at the boundary. Mathematically, one can consider Stokes equations for the liquid 
(variable (x,z) = (xi, x<2, z), velocity field u = (v,w) = (vi,V2,w)): 

-An + Vp = 0, divu = 0, z > 0, (1.3) 
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set above a flat surface {z = 0}. This flat surface replaces the rough hydrophobic one, 
and is composed of areas on which the fluid satisfies alternately perfect slip and no-slip 
boundary conditions. In the simplest models, these areas of perfect slip and no-slip form a 
periodic pattern, corresponding to a periodic pattern of humps and bumps. That means that 
the impermeability condition w = at {z = 0} is completed with mixed Dirichlet /Navier 
conditions: 

d z v = in n, v = in Q c (1.4) 

where Slcl 2 corresponds to the zones of perfect slip and Sl c to the zones of no-slip. The 
whole issue is to design SI so that the energy J (SI) associated with this problem is minimal 
for a fixed fraction of slip area. Unfortunately, this optimization problem (Stokes operator, 
periodic pattern) is still out of reach. That is why we start with the simpler equations 



(1.1) (still difficult, and interesting on their own!). Note that they can be seen as a scalar 



version of (1.3)-(1.4), replacing the Stokes operator by the Laplacian, and the Navier by 



the Neumann condition. Using the classical characterization of (—A) 1 / 2 as the Dirichlet-to- 



Neumann operator leads to system (1.1) and energy (1.2). Again, we stress that the methods 



developed in our paper may be useful in more elaborate contexts. 



If the fractional laplacian in (1.1) is replaced by a standard laplacian (which leads to the 
classical Dirichlet energy), this problem is well-known and described in detail in the book 
|10j by Henrot and Pierre (see also |14j). In this case, one can show that a smooth domain SI 
minimizing the Dirichlet energy under the constraint = 1 is a disc. A standard proof of 
this result has two main steps: 

1. One computes the shape derivative associated with the Dirichlet energy for the laplacian. 
This leads to the following result: if is a minimizer of the Dirichlet energy and mq is 
the solution of the associated Euler-Lagrange equation, then d n u is constant on dSl. 

2. One analyzes an overdetermined problem. More precisely, the idea is to prove that if 
there exists a function u solving 

— Ait = 1 in SI, 
u = on dSl, 
d n u = c on dSl, 

then all the connected components of SI are discs. This second step is achieved thanks 
to the moving plane method. 

Our goal in this article is to develop the same approach in the context of the fractional 



laplacian, showing radial symmetry of any smooth minimizer SI of (1.2). Accordingly, we 
state two main results 

Theorem 1. Assume that there exists an open set !Jci 2 such that SI has C°° regularity, 
\Sl\ = 1, and 

J (SI) = inf J(U). 

r/cM 2 , 
\u\=i 



Let uq be the solution of the associated Euler-Lagrange equation (1.1), and let n(x) be the 



outward pointing normal at x G dSl. Then the following properties hold: 
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1. For all x G d£l, the limit 

un{y) 



lim 



y^>-x,yeU \{y — x) ■ n(x)\ 1 / 2 
exists. We denote it by dn^ 2 un(x). 
2. There exists a constant c > such that 

d l J 2 u n {x) = c Vx G dQ. 

Theorem 2. Let !icM 2 be aC°° open set such that the system 

(-A) 1/2 u = 1 inU, 

u = in n c , (1.5) 
dl/ 2 u = c on dQ. 

has at least one solution. Assume that fi is connected. Then Q is a disc. 

Of course, Theorems [T] and [2] imply immediately the following: 
Corollary 3. Let Qct 2 be a C°° connected open set such that = 1, and 

J(U) = inf J(U). 

UcM 2 , 
\U\=l 

Then Q is a disc. 

Let us comment a little on our two theorems. The main point in proving Theorem [l] (and 
one of the interests of our paper) is to compute the shape derivative of J(f2). We prove in 
fact the following: 

Proposition 4. Let f G C°°(IR 2 ), and let 

J/(fi):= inf (- ((-A) l / 2 v,v\ -[ fv). 

Let £ G C^°(M 2 ) ; and let (4>t)teM. be the flow associated with £, namely 

4>t = C(&), 00 = Id. 

Let Q be an open set with C°° boundary, and let uqj be the unique minimizer of Jf(Q), 

(-A) 1 /V/ = / m SI, 
uqj = in Sl c . 

1/2 

Then d n uqj exists, and there exists an explicit constant Cq such that 
d 



i.e. 



dt 



Jf(</>m)\t=o = C / (d^unjfC ■ n da. 
Jan 
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Notice that this result does not require the connectedness of Q. For our special case / = 1, 
Theorem [T] implies that the value of the fractional normal derivative is the same on all the 
boundaries of the connected components of O. 

We stress that, due to the non-locality of the fractional laplacian, this shape derivative 
is hard to compute. In the case of the classical laplacian, it is obtained through integration 
by parts, which are completely unavailable in the present context. The idea is to bypass the 
nonlocality by using an asymptotic development of near the boundary. 

As regards Theorem [2j it is deduced from an adaptation of the moving plane method 
to our fractional setting. Again, this is not straightforward, as the standard method relies 
heavily on the maximum principle and Hopf's Lemma, which are essentially local tools. To 
overcome our non-local problem, we use appropriate three-dimensional extensions of u, to 
which we can apply maximum principle methods in a classical context. Note however that 
we need to assume that f2 is connected: this hypothesis is precisely due to the nonlocality of 
the fractional laplacian. 

We conclude this introduction by a brief review of related results. Let us first mention that 

1/2 

the condition d n u = c appears in other problems related to the fractional laplacian. In [6J, 
Caffarelli, Roquejoffre and Sire consider a minimization problem for another energy related 

1/2 

to the fractional laplacian, and they prove that minimizers satisfy the condition d n u = c on 
d{u = 0}. However, we emphasize that the issues of the present paper and those of [6J are 
rather different. The problem addressed in [6J is essentially a free-boundary problem (i.e. Q 
is not given, but is defined as {u > 0}), and therefore questions such as the regularity and the 
non-degeneracy of u, and the regularity of the free boundary, are highly non trivial and are 
at the core of the paper [BJ. Here, our goal is not to investigate these questions, but rather 
to derive information on the shape of Q. 

As regards our adaptation of the moving plane method, it relates to other results on the 
proof of radial symmetry for minimizers of nonlocal functionals: see for instance |13j on local 
Riesz potentials 

= / i ia^T 

Jn \ x - y\ 

or [5] on the radial symmetry of solutions of nonlinear equations involving A 1 / 2 , where A 
is the Dirichlet laplacian of a ball in W 1 . Note that in these two papers, "local" maximum 
principles are still available, which helps. Further references (notably to article [3]) will be 
provided in due course. 

Let us eventually point out that the question of shape optimization for J(f2) has been 
addressed, in a slightly different context, by Lopes and Maris in |12j . Their result is the 
following: 

Proposition 5. Let s £ (0, 1) and assume that F, G : M — > K are such that u i— )■ F(u), u \— )■ 
G(u) map H s (M. d ) into L 1 (M' i ). Assume that u E H s (M. d ) is a solution of the minimization 
problem 

Minimize E(u) := [ \£\ 2s \u(0\ 2 d£ + f F(u) 

JR d JR d 

under the constraint / G(u) = X. 

Then u is radially symmetric. 
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Our problem does not strictly fit in theirs, although the energy J can be written as 



J(U)= inf - / \£\\u^dZ- / u. 

Indeed, the functions 

u i y zij u i y l u >o 

do not map ij 1 / 2 (M 2 ) into L 1 (M 2 ). However, looking closely at their proof, it is likely that 
their arguments can be extended as such to the energy J(O), which would imply that un is 
radially symmetric if f2 is a minimizer of «/($!). Notice that our result is different from theirs: 

• We prove that if fi is connected, then it is a disc, while their result implies that it is 
either a disc or a ring; 

• Combining Proposition [5] and Corollary [3j we infer that if is a mimimizer of J, then 

is a disc. Indeed, Proposition [5] implies that has radial symmetry, and therefore 
its boundary has C°° regularity. Hence f2 is either a disc or a union of concentric rings. 
The latter case is excluded thanks to Theorem [T] and a slight modification of Theorem 
[2] (see Remark [20]). 



We also emphasize that our method relies on more local arguments: indeed, the techniques 
used in |T2] rely on the Fourier transform. Hence the arguments developed here might be 
generalized more easily to fractional operators with non-constant coefficients. 

The plan of our paper is the following: Section [2] collects more or less standard results on 
the fractional laplacian, which will be used throughout the article. Special attention is paid 



to regularity properties of solutions of (1.1), that we deduce from regularity results for the 
Laplace equations in domains with cracks. Section [3] is devoted to the proof of Theorem [TJ 
and Section [4] to the proof of Theorem [2} 

2 Preliminaries 

2.1 Reminders on the fractional laplacian 

We remind here some basic knowledge about (—A) 1 / 2 , see for instance [15]. We start with 
Definition 6. For any f G 5(IR n ), one defines (—A) 1 / 2 / through the identity 

(-a)V 2 /(o = iei/(o- 

Note that g = (—A) 1 / 2 / does not belong to 5(M n ) because of the singularity of |£| at 0. 
Nevertheless, it is C°° and satisfies for all 

sup(l + |x| n+1 )| 5 ( fc )(x)| < +00 

This allows for a definition of (—A) 1 / 2 over a large subspace of 5'(M n ), by duality. We shall 
only retain 

Proposition 7. (—A) 1 / 2 extends into a continuous operator from H 1 1 2 (W 1 ) to if~ 1//2 (M n ). 
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This is clear from the definition. 

It is also well-known that (— A) 1 / 2 can be identified with the Dirichlet-to-Neumann oper- 
ator, in the following sense (writing (re, z) G W l x R the elements of W l+1 ): 

Theorem 8. Let u G H 1 / 2 ^ 11 ). One has (— A) l / 2 u = — <9 2 ?7| 2= o, where U is the unique 
solution in 

H 1 {R r l +1 ) := {U G L 2 oc (R™ +1 ), VC/ G L 2 (M r | +1 )} 

of 

-A x , z U = inR r l +1 , U\ z=0 = u. 

Let us remind that the normal derivative d z U\ z= o G -ff~ 1 / 2 (IR n ) has to be understood in 
a weak sense: for all G iT^R" 4 " 1 ), 



< 9 2 C/| 2=O ,70>= - / W- 



where 7 is the trace operator (onto H 1 / 2 ^ 71 )). It coincides with the standard derivative 
whenever U is smooth. 

We end this reminder with a formula for the fractional laplacian: 

Theorem 9. Let f satisfying J R „ pj^^fi dx < +00, with regularity C 1,<E , e > 0, over an open 
set fL Then, (—A) 1 / 2 / is continuous over f2, and for all x G f2, one /ias 

(-A) 1//2 /(x) = dPv/ f , (x) ~fff <fr (2.1) 

_ /■ /W-/M-v/W-(.-»)i^ 

7iR2 |x-y| n+1 
/or some Ci = C\(n) and any C > 0. 

2.2 The Dirichlet problem for (—A) 1 / 2 . Regularity properties (2d case). 



In view of system ( 1.1 ), a key point in our analysis is to know the behavior near the boundary 



of solutions to the following fractional Dirichlet problem: 

{-A) 1/2 u = f on O, 



(2.3) 
u = on fi c , 



where O is a smooth open set of R 2 and / G C°°(IR 2 ). Note that in system (2.3), we prescribe 
the value of u not only at <9f2, but in the whole O c . This is reminiscent of the non-local 
character of (—A) 1 / 2 : remind that u = U\ z= o, where u satisfies the (3d) local problem 

-AU = in z > 0, , s 

2.4 

d z U = -f over Q x {0}, U = over tt c x {0}, 

whose mixed Robin/Dirichlet boundary condition must be specified over the whole plane 
{z = 0}. 
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We were not able to find direct references for regularity properties of problem ( |2.3[ ), 
although the C ' 1 / 2 regularity of u and the existence of dn 2 u are evoked in [U |6l [9]. In 
particular, we could not collect information on transverse and tangential regularity of the 
solution u near d£l x {0}. We shall use results for the Laplace equation in domains with cracks, 
in the following way. Let 9 £ C^°(]R 3 ), odd in z, with 9(x, z) = z for x in a neighborhood of 
Q, and \z\ < 1. Then, V(x,z) := U(x,z) + 9(x, z)f(x) satisfies 

-Ay = F in z > 0, 

(2.5) 

d z V = over SI x {0}, U = over fi c x {0}, 

where i 7 := —A x ^ z [9f) is smooth, odd in z, and compactly supported in M 3 . We then extend 
V to {z < 0} by the formula V(x, z) := — V(x, —z), z < 0. In this way, we obtain the system 



-AV = F in t 3 \(S]x {0}) 
d z V = at (fix {0}) 



(2.6) 



which corresponds to a Laplace equation outside a "crack" fi x {0} with Neumann boundary 
condition on each side of the crack. We can now use regularity results for the laplacian in 
singular domains, such as those of |7J. First, note that V is C°° away from fi x {0}, by 
standard elliptic regularity. Let now T be a connected component of <9fi, and ip a truncation 
function such that (p = 1 in a neighborhood of T, with Supp ip H (90 \ T) = 0. Then, pV still 
satisfies a system of type (2.6), with fi replaced by Supp p, and F by F — [A, yj]V (which is 
still smooth). We can then apply [7J Theorem A. 4. 3], which leads to the following 

Theorem 10. Let V be a connected component of d£l. We denote by (r,6) polar coordinates 
in the planes normal to T and centered on T, and by s the arc-length on T, so that 

R 3 \ ((Supp^nfi) x {0}) = {(s,r,9), s E (0,L), r > 0, 9 £ (-7r,7r)}. 



Then, the solution V of ( |2.6[ ) has the following asymptotic expansion, as r — > 0: for any 
integer K > 0, 

K 

v = J2r 1 / 2+k ^ k (s,9) + V reg . K + U rem , K (2.7) 

3=0 

where: 

• the coefficients ^ k are regular over [0,L] x [— tt, 7t], 

• U re g : K is regular overM?, 

• the remainder U rem ^K satisfies d^U rem ^K = o(r K ~^^ +1 ^ 2 ) for all (3 6 N 3 . 
Setting 

^ k ( s ) ■= $ fc ( s ,7r), u rem , K (x) ■= U remtK (x,0 + ), u re g )K {x) := U reg , K (x,0 + ), 



we can get back to the solution u of (2.3) and obtain the asymptotic expansion 

K 

u = ^2r l/2+k i) k (s) + u reg)K + u rem>K . (2. 
Such formulas will be at the core of the next sections. 
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3 Shape derivative of the energy J(Q) 

This section is devoted to the proof of Theorem [TJ In fact, Theorem [T] is an immediate 
consequence of Proposition [4} Indeed, assume that £1 is a minimizer of J under the constraint 
= 1. The theorem of Lagrange multipliers then implies that there exists a constant A such 
that for any group of diffeomorphisms [<t>t)t&Wi 

~(J(<^)) + A|0 t (fi)|) = at t = 0. (3.1) 



( • n da, 



Assume that (4>t)teR is the flow associated with £ G Cq°(M ). Then it is proved in [lOj that 

'd\Mm)\ 

dt J | t=0 7an 

while Proposition [4] implies that 

d : j(0 t (o))| t=o = Co / (c-nXay 2 ^) 2 ^. 



(it 



an 



Thus (3.1) becomes 



3A G R, VC G C °°(IR 2 ), ^(C • n) (W 2 u Q ) 2 + ~) = 

1 /2 

Since ( is arbitrary, we infer that d n uq is constant on O. Moreover, since > on $7 by 
the maximum principle, the constant is positive. 

Remark 11. To be fully rigorous (see fHJIj). we should in fact compute the differential of the 
application 

r] G Cl{R 2 ,R 2 ) i — y J{(I + r))Q) 

at the point rj = 0, and apply the theorem of Lagrange multlipliers to the above functional. 
However, the proof seemed more legible with a family of diffeomorphisms (<pt)teR> an d with a 
classical derivative with respect to t. Let us also mention that both approaches are equivalent 
as soon as we have proved that the application rj \— > J ((I + tj)CI) is differentiate. Indeed, 



\t=o 



is the Gateaux derivative of J ({I + rf)Q) in the direction £. 

The differ entiablity of rj \— > J {{I + rf)Q) is easy to obtain: we merely prove that 

r) G C\ (M 2 ,]R 2 ) t->Vr,e H 1/2 (R 2 ) 

is differentiate, where = o (/ + 77) and u v = Urj+^Qj is the solution of the Euler- 
Lagrange equation associated with (I + The proof goes along the same lines as the one 

of Lemma 12 below. Furthermore, the variational formulation of the Euler- Lagrange equation 



implies ( see formula ( |3.9| ) ) 

J/((/ + 77)n) = -i / v v fo(I + r] )det(I + V V ). 
The differentiability property follows. 
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We now turn to the proof of Proposition [4j In the case of the classical laplacian, the shape 
derivative of the Dirichlet energy is well-known and is proved in the book by Henrot et Pierre 
|10j . Let us recall the main steps of the derivation, which will be useful in the case of the 
fractional laplacian. Let 



It (SI) := inf f - f |Vu| 2 - [ fu) 



For all C S C 1 Pi W 1,0 °(M. 2 ), consider the flow <f> t associated with (. Then for all t E R, (fa is a 
diffeomorphism of M. 2 . We recall the following properties, which hold for all t £ R: 



rt 

<Po = C, |det(V&(x))| =: j(t, x) = exp ( / (div C)(^(z)) ds 



o 



-l 



(3.2) 



7 f 



-C, IdetCV^ 1 )! =i(-t,x). 



|t=o 

The last line merely expresses the fact that (jx^ 1 = <ft-t, for all tel. 

For t > 0, let fit = (f>t(Q), and let be the solution of the associated Euler-Lagrange 
equation, namely 

-Aw t = f in fit, 
wt = on dSlf 

Eventually, we define z t := wt o (fit- 
It is proved in [10] that 



at J i t=0 1 J a 



(C • n)(<9 n u> ) da. 

\t=o z ^en 

Indeed, since tUf solves the Euler-Lagrange equation, we have 

I f (n t )=-- J f Wt = -- [ z t (y)f o<fr t (y)j(t,y) dy. 

Differentiating zt = wt° <fa with respect to t, we obtain 

zt = w t + 4>t ■ Vro ( , 

and thus in particular 

wo + C • n d n wo = on <9Q. 
The Euler-Lagrange equation yields 

— Aibo = in O. 

Gathering all the terms and using the fact that Wq^qq = 0, we deduce that 



(^/(^J^ o = -^(/(^o + C-VtooJ + CC-V/H + ZdivC^o) 

-w Aw + div (CM))) 



|*=0 

1 

~2 
1 



wqAwq + / wod n woda 
<I JdCl 

(C • n)(d n w ) 2 da. 
an 
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Therefore the shape derivative of If is similar to the one of J/, the fractional derivative being 
merely replaced by a classical derivative. 

Unfortunately, the proof in the case of the classical laplacian can only partially be trans- 
posed to the fractional laplacian. Indeed, several integration by parts play a crucial role in 
the computation, and cannot be used in the framework of the fractional laplacian. 

We use therefore a different method to estimate dJf(Qt)/dt. The main steps of the proof 
are as follows: 

1. As above, we introduce ut = UQ t j and vt = ut° 4>t- We derive regularity properties and 
asymptotic expansions for vt, from which we deduce a decomposition of no and uq. 

2. In order to avoid the singularities of uq near d£l, we introduce a truncation function Xk 
supported in £1, and vanishing in the vicinity of the boundary. Using the integral form 
of the fractional laplacian, we then derive an integral formula for an approximation of 
dJ f(£lt) I 'dt involving uq, uq and Xk- 

3. Keeping only the leading order terms in the decomposition of uq and uq, we obtain 

1/2 

an expression of dJf(VL t )/dt in terms of d n uo, and we prove that this expression is 
independent of the choice of the truncation function Xk- 

4. We then evaluate the contributions of the remainder terms to the integral formula, and 
we prove that they all vanish as k — > oo. 

Before tackling the core of the proof, let us introduce some notation: 

• We denote by F±,- ■ ■ the connected components of dtl, and we parametrize each 
T{ by its arc-length s. We denote by Lj the length of IY 

• The number r£l stands for the (signed) distance to the boundary of Q. More precisely, 
\r\ is the distance to the boundary, and r > inside Q, r < outside S7; 

• We denote by Ut the three dimensional extension of u t in the half-space, i.e. the function 
such that 

-AU t = on R%, U t \ z=0 = ut. 
The three-dimensional function Vt is then defined by 

V t (x,z) = U t {(f)t(x),z) x€R 2 ,z€R, 

so that V t \ z =o = v t . 

• Derivatives with respect to t are denoted with a dot. 

3.1 Regularity of u and w and expansions 

We start with the following lemma 

Lemma 12. For all t in a neighbourhood of zero, 



u t G # 1/2 (M 2 ), u t G #~ 1/2 (R 2 ) 



(3.3) 
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The proof is rather close to the one of Theorem 5.3.2 in [TO]. In order to keep the reading 
as fluent as possible, the details are postponed to the end of the section. The idea is to prove 
that Vt solves a three-dimensional elliptic equation with smooth coefficients. The implicit 
function theorem then implies that t i— > Vt G H 1 is C 1 in a neighbourhood of t = 0. Since Vt 
is the trace of Vt, t \— > vt G H 1 ! 2 is also a C 1 application. Eventually, the chain-rule formula 
entails that iit G H^ 1 / 2 . 

We also derive asymptotic formulas for Vt and vt in terms of r: we rely on the results in 



the paper by Costabel, Dauge and Duduchava [7j, and we use the notations of paragraph 2.2 



(see also Theorem 10 of the present paper). We claim that there exists ipo,ipi,i^2 G C l {d£l), 
ui,u a G W 1 ' 00 ^ 2 ) such that 

u = Jr^tpo(s) + ui(x), (3.4) 



uq = l r>0 ^=Vi(s) + \Av02(s) + u 2 (x), (3.5) 



where 



^o(s) = ay 2 u (s), (3.6) 
Ms) = -\(-n(s)Ms)- (3-7) 
Moreover, there exists 5 > such that 

u t €L 00 ((-<y,5),L 1 (R 2 )). (3.8) 
These decompositions and the regularity result will be proved at the end of the section, after 



the proof of Lemma 12 



3.2 An integral formula for an approximation of dJf(Vt t )/dt 



We first use the Euler-Lagrange equation (1.1) in order to transform the expression defining 
Jf(p,t). Classically, we prove that the unique minimizer u of the energy 

in the class {v G if 1//2 (IR 2 ), v\qc = 0} satisfies 

((-A) 1/2 u,v) -[ fv = Vv G H 1/2 (R 2 ) s.t. v la c = 0. 

Choosing v G Cg°(fi), we infer that (— A) 1 / 2 ^ = / in $7. Since U|^c = 0, we infer that 
u = UQ t j = ut, and in particular 



((-A) 1 / 2 n t ,n t )= / fu t . 



nil 

The identity above yields 

Jf^t) = ~\ ! fut = -\ I fu t . 
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My)! ° My)j(t,y) dy. 



Changing variables in the integral, we obtain 

Since 1 1— > Vt € H 1 / 2 is differentiable, 1 1— > Jf(Qt) is C 1 for t close to zero, and 
d 

|t=0 2 

Now, for A; G N large enough, we define Xfc £ C^°(M 2 ) by Xfc( a; ) = x(kr), where x G C c 
and x(p) = for p < 1, x(p) = 1 for p > 2. Then, since uq = vq, 



(3.9) 



(^/(«t)) = ^(«0/ + C • V/^o + ^o/div C). 



|t=0 



•— lim 

2 fc— >oo 



Xfc(*o/ + w div (C/)) 



- lim ( / Xfc^o div (£/) + 

^ fc^oo W R 2 

- lim ( / XkU div (C/) + 



d_ 
df 



Xkfvt 

\t=o 

Mfxk) o ^ x j{-t,-) 



|t=0, 



For fixed fc and for i in a neighbourhood of zero, there exists a compact set isT^ such that 
Kk <£= O and Suppx/c C -Kfc. Since Ut E Lf(Ll.) according to (3.8), we can use the chain 
rule and write 



j t I ut(fXk)°(f> t VC-V) 



|t=o 



/ (^o/Xfe - u fC ■ Vxfc - uoXfcC • V/ - uoXfc/div ()■ 
Jm. 2 



Using the decomposition (3.4) together with the definition of \ki we deduce that 

C 



< C 



/r+k\x {kr) \ dr < 



Notice also that 



(-^oXfcC • V/ - noXfe/div Q 



uoXkdiv (/C)- 



We now focus on the term involving uq; since (— A) 1 / 2 uo = / on the support of Xk, we 
have 



UofXk 



"oXfc(-A) 1/2 n 
«o(-A) 1/2 («oXfc)- 



Notice also that Xk{~ A) ' (uq) = 0. Indeed, iit is smooth on Kf. for t small enough (see 
for instance (3.27) below). Hence for x S K^, the integral formula (2.2) makes sense and we 
have, using (3.8), 

(-A) 1 / 2 -u (x) 



Ci 



uq(x) - 7io(y) - Vwo(^) • {x - y)t\x- y \<c , 
i y 

\x — y\° 

d_ f u t {x) - u t (y) - Vu t (x) ■ (x - y)l\ x - y \<c 
dt J R 2 \x-y\ 3 



dy 



0. 



|t=o 
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Eventually, we obtain 

/ u fxk = uo (-A) 1 
Jm. 2 Jr 2 1 

Gathering all the terms, we infer eventually 



/2 



Xk 



J f (Slt 



\t=o 



— lim 

2 k— >oo 



"0 



U . 



(-A) 1 / 2 ,Xfc 



n . 



Let us now express the right-hand side in terms of the kernel of the fractional laplacian. Using 



the integral formula (2.2) together with the expansion (p. 5ft, we infer that 



uo(y){xk{x) - xk{y)) - u (x)Vxk{x) • (x - y)l\ x - y \ 



<c 



\x - y\ 



dy. 



The value of the integral above is independent of the constant C . Therefore the shape 
derivative of the energy Jf is given by 



d 
dt 



Ci 

Jf{O t )\t=o = — lim wnere 

Z k—>-oo 



u (x) 
\x — y\ : - 



{uo(y)(Xk(x) - Xk{y)) - uo(x)Vxk{x) ■ (x - y))l\ x - y \<c} dx dy. 



(3.10) 



3.3 Asymptotic value of Ik 



We now evaluate the integral 1^ defined in (3.10). There are two main ideas: 



We prove that the domain of integration can be restricted to a tubular neighbourhood of dO, 



(see lemma 13). 



• Since the integral Ik is bilinear in uo,uq, we replace uq and iio by their expansions in (3.4), 



(3.5). The leading term is obtained when uq and iio are replaced by the first terms in their 



respective developments. We prove in the next subsection that all other terms vanish as 
k — > oo. 

We begin with the following Lemma (of which we postpone the proof): 
Lemma 13. For 6 > 0, let 

T 5 := {iEl 2 , d(x,dQ) < 5}. 



Choose C = | in the definition of Ik (3.10). Then there exists a constant C$ such that for 
k > 5/5, 



r u (x) 
'(T s xT s y \x-y\ l 



{uo(y)(Xk{x) - Xk(y)) - u (x)Vxk{x) ■ (x - y))l\ x - y \<s/2} dx dy 



< 



Vk' 
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We henceforth focus our attention on the value of the integral on T§ x Tg. Replacing no 
and uq by the first terms in the expansions (3.4), (3.5), we define 



k •- 



T s xT s \X-V\ 



^l(s')^j=-(x(kr) - x(kr')) 

lr>0 



?Pi(s) J ^kx(kr)n(s) • (x - y)l\ x - y \<s/2 



dxdy. (3.11) 



There is a slight abuse of notation in the integral above, since we use simultaneously carte- 
sian and local coordinates. In order to be fully rigorous, r,r',s,s' should be replaced by 
r(x), r(y), s(x), s(y) respectively. However, the first step of the proof will be to express the 
integral it in local coordinates, and therefore we will avoid these heavy notations. 

In fact, the computation of the limit of if is much more technical than the estimation of 
all other quadratic remainder terms, which we will achieve in the next subsection. We now 
prove the following: 

Lemma 14. There exists an explicit constant C2, independent of x, such that 

f Jj, 



lim lim if = C2 7^ / ipo(s)ipi(s) ds. 

<5->0 fc-s>oo ' Jq 



i=l 



Proof. Several preliminary simplifications are necessary: 

• We use local coordinates instead of cartesian ones, i.e. we change x into (s, r) and y into 
(s',r'). Since |r|,|r'| < 5, the jacobian of this change of coordinates is, for 5 > small 
enough, 

|l + m(s)| |l +r'K(s')\ = (l + m(s)) (l + r'K(s')) , 

where k is the algebraic curvature of <9f2. We refer to the Appendix for a simple proof. Notice 
that this jacobian is always bounded. 



We write Tg 



U^Tj, where 



T} = {x€R 2 , d(x,Ti) < 5}. 



If 8 is small enough, TgDT^ 



i s i i± s — v for i 7^ j, and \x — y\ is bounded from below for x £ Tg, y £ T$ 
by a constant independent of 5. Hence, for i ^ j, 



L 



^o('S)v / ^+ 

3 



TixT? \X~y 



^i{s')-^=-{x{kr) - x{kr')) 
l r >o 



^i{s) JL ^kx' '(kr)n(s) ■ (x - y)l\ x - y \<s/2 



dx dy 



< c 

< C 

< C5 2 



TjxTl 



(0,5) 2 



L r,r'>0" 



dx dy 



dr dr' 



Therefore, in the integral defining 1%,, we replace the domain of integration Tg x Ts by U-L-^Tg x 
Tl, and this introduces an error term of order 5 2 . 
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In local coordinates, we write TJ as (0,Lj) x (—5,5). We replace the jacobian 

(l + r«(s)) (l + r'«;(s / )) 

by (1 + re(s)) 2 . Since s,s' £ (0, Lj) (i.e. x and y belong to the neighbourhood of the same 
connected component of d£l), this change introduces error terms bounded by (3.18), (3.20), 
which vanish as k — > oo and 5 — > 0. More details will be given in the fourth step for similar 



error terms; we refer to Lemma 15 



We evaluate \x — y\ in local coordinates for x, y £ TJ. We have 

x = p(s) — rn(s), 
V = p( s ') — rn(s), 

where p(s) £ M 2 is the point of dtt with arc-length s. Since the boundary T{ is C°°, 

p( s )-p( s ') = ( s -s')T(s) + 0(\s-s'\ 2 ), 

where r(s) is the unit tangent vector at p(s) £ 90. Using the Frenet-Serret formulas, we also 
have 

n(s) - n(s') = -(a - s')k(s)t(s) + 0(|s - s'\ 2 ), 
where k is the curvature of Tj. Gathering all the terms, we infer that 

x-y = (s- s')(l + «;(s)r)r(s) + (r - r')n(s) + 0(|s - s'\ 2 + \r- r'\ 2 ), (3.12) 

so that 



\x - y\ 2 = {s- s') 2 (l + n{s)r) 2 + (r - r') 2 + O (|a - s'| 3 + \r - r'| 3 ) 



(3.13) 



and 



\x - y\~ 3 = ((a - s') 2 (l + t(s)r) 2 + (r - r') 2 ) 3/2 (l + 0(\ 



s — s'\ + \r — r'|)) . 



In particular, there exists a constant C such that 



1 



< 



C 



A2^3/2' 



F-y| a ((s-s') 2 + (r-r') 2 ) 

and replacing |tc — y|~ 3 by ((s — s') 2 (l + ^(s)r) 2 + (r — r') 2 ) 3 ^ 2 generates yet another error 
term bounded by d3l8b, 13.201). 



We replace the factor V'i( s ') by Vi ( s ) ? this also leads to an error term of the type (3.18). 



Using (3.12), we infer that 

n(s) ■ (x-y) =r - r' + 0(\s - s'\ 2 + \r - r'\ 2 ). 

The second term in the right-hand side of the above equality gives rise to an error term of 
the type (13.20}. 
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The last preliminary step is to replace the indicator function l\ x -y\<s /2 m (3.11 ) by a quantity 
depending on s,s',r,r'. Using the asymptotic development (3.13) above, it can be easily 
proved that there exists a constant c such that 



l \x-y\<8/2 - L (s-s') 2 (l+K(s)r) 2 +(r-r') 2 <8 2 /4 



- 1 (l-c8)l<\x-y\<(l+cS)l ■ 



Therefore the substitution between the two indicator functions yields an error term bounded 
by 

N 



< 



E 

i=i 

N 

E 



dx dy 

TjxTj \ x - y\' 



|Vxfc(aO|l a _ 



8=1 

< Cm 



\VXk{x)\ dx 



(l-cS)%<\x-y\<{l+cS)$ 



dz 

02 1 (l-c5)|<|2|<(l+c5)f 



l + c6 



< C8. 



1 - cS y 

As a consequence, at this stage, we have proved that 



2\3/2 



£, , , 

^ Jo J-S J(0M) 2 ((a - s') 2 (l + n{s)r) 2 + (r - r') 2 ) 

*° -(x(fcr) - x(kr')) - -^k X '(kr)(r - r / )l( s _ s ')2(i +Kr )2 +(r _ f ,, ) 2< <5 2 / 4 



(3.14) 
ds ds ' dr' dr 

+ o(^ + J | M ). 

We now evaluate the right-hand side of the above identity. We first prove that the term 
involving x'{kr) does not contribute to the limit, due to symmetry properties of the integral. 
This was expected, since this term had a vanishing integral in the beginning; its only role was 
to ensure the convergence of We then focus on the term involving x(kr) — x(kr'), and we 
prove that its asymptotic value is independent of x- 

First, since the integral (3.14) is convergent, we have 

fS 



N p8 j-8 

e3=EjW„l, 



-8 J(0,Li) 2 



l \r-r'\>e ' ' " 



Therefore, up to the introduction of a truncation, we can separate the two terms of (3.14). 
We have in particular 

3/2 kx {kr){r - r )l (s _ s , ) 2 (1+Kr) 2 +(r _ r , ) 2< 5 2 /4 dr 



-5 ((s - s') 2 (l + K(s)r) 2 + (r- r') 2 ) 

S ~ r Vo(s)V>l(s)l| f |>e(l+^(s)) 2 



-k x \kr)£ 1 



(s-s') 2 (l+Kr) 2 +. 



? 2< 5 2/ 4 d^. 



(3.15) 



-8-r {( S - S') 2 (l + K(s)r) 2 + ef /2 

Notice that the above integral only bears on the values of £ such that |£| < 5/2. On the other 
hand, for all r such that x'{kr) ^ 0, we have 1/k < r < 2/k, so that if k > 4/5, 



5 — r > 5 



2 5 

k > 2' 



r < -8 < --. 

2 
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Hence the integral (3.15) is in fact equal to 

/ ( \ i ( \ft . i ^^2, //, \ f 5/2 ^ 1 |ei>e 1 (»-*') a (i+'w) a +f a <* , /4 At 
Vo{s)ipi{s){l + rK(s)) kx{kr) — i—r d£. 

J -5/2 {{s-s') 2 {l + K{s)r) 2 + e) f 

Since the integrand is odd in £, the integral is identically zero for all e > and for all S, k 
such that kS > 4. There remains to investigate the first term in (3.14), namely 

/ f t^rnm + ™w) 2 (x(fcr) - x(kr>)) jf ds , dr , dr 

7(0,5)2 J(0M) 2 {{s-s') 2 {l + K{s)r) 2 + {r-r') 2 f 12 V r' 1 
We first symmetrize the integral by exchanging the roles of r and r'. We have 

r -{l + rK{s)f - J-(l + r'K(s)) 2 



r 



(l + r K (s)) 2 + 



((s - s') 2 (l + K(s)r) 2 + (r - r') 2 ) 3/2 ((s - s') 2 (l + K(s)r') 2 + (r - r') 2 ) 
O 



2\3/2 



v ((a - s') 2 + (r- r') 2 ) 



,2^3/2 



The term 



/ / |^ (s)^i(s)(x(>r) - x(kr'))\ \ — 
J(o,8) 2 J{o,U) 2 V r 



r ((« - s') 2 + (r- r') 2 ) 



2^3/2 



ds (is' dr' dr 



/(0,<5) 2 ^(CL,) 2 

is bounded by an error term of the type (3.18), and is therefore 0(lnk/Vk) for all e > 0. As 
a consequence, 



lim(3.16) 



(0,5) 

In k 



, ± rjMfMM ~ ^\ r 'P r Z^dsds'dr'dr (3.17) 

zJ(0M) 2 ((s-s') 2 (l + K(s)r) 2 + (r-r') 2 ) 3/2 VrV 



+0 

We now compute the integral with respect to s' £ (0, Lj). Setting 

dz 



m ■= [ 
Jo 



o (1 + z 2 ) 3 / 2 ' 



we have 



/' 

Jo 



ds' 



2^3/2 



((s - s') 2 (l + K(s)r) 2 + (r - r') 2 ) 

(Li - s)(l + k(s)i 



(r — r') 2 (l + fc(s)r) 



+ 



?(1 + K(s)r) 



r — r 
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Inserting this formula into the integral above and changing variables, we obtain 



(3.17) 



, x{p) - x(p') 



\l l ' Ms)Ms)f^ff4 (i + i 

2 J(o,kS) 2 Jo {p-P)VPP y k ' 



k{L l -s)(l + K (s)p/k) \ + / ks(l + K (s)p/k) \ 
\P~P'\ J \ \P~P'\ J 



ds dp dp' . 



Since 



o Jo 



x(p) - x(p') 



(p- P')\/W 



dp dp' < oo, 



and < 0(C) < <f>(+oo) < oo V£ > 0, 
using Lebesgue's theorem, we infer that for all 5 > 0, 



k— >oo 



lim (3.17) = <f)(+oo) / ifo(s)i/>i(s)ds 



x{p) - x(p') 



o ./o (p-p')vpp' 



=dp dp. 



There only remains to prove that the integral involving x is m fact independent of x- We 
have 



In 



x(p) - x(p') 



o (p-p')VPP' 

oo roo rl 



j dp dp 



poo poo pi 

/ / / X'(rp+(l-r)p') 
Jo Jo Jo 



,-^dr dp dp' 

\fp~P 1 
dr dp dz 



foo poo pi 

/ / x '(p(r+(l-r)z)) 
z=p'/p JO JO JO V z 



oo rl 



1 



dr dz 



lo (z-l)Vi 
Gathering all the terms, we infer that 



o t + (1-t)z yfz 
In z 

dz. 



lim 4 = / o 0(+oo) V / VoV'i + O (S\ lnS\) 

k— >oo * — » 



i=l 



Passing to the limit as 5 — > 0, we obtain the result announced in Lemma 14 



□ 

3.4 Evaluation of remainder terms in the integral Ik 

> We start with the proof of Lemma 13 . The idea is to divide M 2 x IR^T^ x T$ into subdomains 



and to evaluate the contribution of every subdomain. 



• For (x,y) £ (T£ n f2) 2 , Xk(x) = Xk(y) = 1 and Vxk(x) = for k > 5 1 , so that the 
contribution of Tg n ft x n £1 is zero. 

• For x E T? n J7 C , lio^) = 0: the contribution of T£ n f2 c x M 2 is zero. 
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IS 



For x £ T% fl Q,y £ Tg D iio(y) = and Vxk(x) = 0, so that the contribution of 
Ifflfix T| n ft c is zero. 

For x 6 T^DO, y G T5, Xfc( x ) = 1 an d Vxk(x) = 0, so that the contribution of the sub-domain 

/ 1 [3 {uo(y)(Xk{x) - Xk{y)) - iio{x)Vxk{x) ■ (x - y)l\ x - y \<5/2} dx dy 

Jr^nnxTs \ x ~ V\ 

< C I - — —r^\u (y)\(l- Xk(y))dxdy. 

k 

side is bounded by 



Since 1 — Xk is supported in f2 c U {d(y, d£l) < 2/k} and |uo(y)| < C ^^ n Q ^ ; the right-hand 

C f 2/k dr C s 



II 3 Jo 



\6-l\ 3 Jo V^'Vk 

For x £ T s , y £ Tg, the integral 

f u (x)u (x) 

JTgxTc \x — y\ 

is in fact supported in 

({d(x, 90) < 2/k} x 30) > 5}) D {\x - y\ < 5/2}. 

It is easily seen that for k large enough (say k > 5/5) this set is empty, and therefore the 
integral is zero. 

There only remains 

1 U ° (yX \ 3 {l o(y)(Xk(x) - Xk(y)) dx dy. 

T s xT£ \X — y\ 

Using the same kind of estimates as for the domain Tg D O x Tg, it can be proved that this 
term is bounded by C^/c -3 / 2 . □ 

> We now estimate the remainder terms on Tg x Tg. We use the following lemma: 

Lemma 15. Let k > 1 and 5 > such that k5 > 5. 
Then the following estimates hold: for 1 < i < N, 

f \Fr 1 / In \ 

/ lr,r'>o /-t, 7: t=R'\ 12 XH X - Xk(l/) dxdy = Ol—=), 

JTixTi Vr'(yfF + vV) \x - y\ 2 \VkJ 



/ I [3 |xfc(«) - Xfe(y) - Vxfe(x) • (x - y)l| x _ w |< tf / 2 | dxdy = 0[y/8+—=), 

Jt i s xti \x - y\ \ v k / 

(3, 

f lkX ' {kr ^ dxdy = 0(5\ln5\), 
JrixTi \x-y\ 



(3.18) 

)■ 

(3.19) 



(3.20) 



I lr>o,r'<o , ^ l3 \Xk(x) - Xk{y)\ dxdy = O ( — ^ J . 

(3.21) 



19 



Proof. Throughout the proof, we use the following facts: 
The jacobian of the change of variables (x, y) — > (s, r, s' , r') is 

(l + r«(s)) (l + r'«(s')) < C; 



Using the expansion (3.13), we infer that there exists a constant C such that 

C 



1 



< 



|as — y| ~ ((s - s') 2 + (r - r ') 2 ) 1 / 2 ' 

For all a > 1, there exists a constant C a such that for all s E (0, Li), for all r, r' S (— (5, 5) 
with r ^ r' , 



r 

Jo 



ds' 



((s - s') 2 + (r - r') 2 )°/ 2 



< c a 



| r _ r /|l-a if a >i, 
I In \r — r'\ I if a = 1. 



(3.22) 



Indeed, for a > 1 

"Li 



((s - s') 2 + {r - r') 2 ) a / 2 



< 



< 

£=2/|r— r'| 



/ 

J& 



dz 



z* + (r — r 



a 



/)2W2 
(l + £2)a/2' 



while for a = 1 



Jo 



ds' 



((s - s') 2 + {r - r') 2 ) 1 / 2 



dZ 



\r — r 
Li 



< 



< 2 In 



^(1 + ^)1/2 

1 dtj 

\ (1 + £ 2 )V2 

— r' | 



We now tackle the proof of (3.18 )-(3.21 ). 



1. Proof of (3.18): we split the domain (T$ x T$) n {r, r' > 0} into four subdomains: 

• < r,r' < 3/fc; 

• < r' < 3/k and 3/fe < r < 5; 

• < r < 3/k and 3/k < r' < 5; 

• 3/k <r,r' < 5. 

Since Xk{%) = 1 for r > 2/fc, it is easily seen that Xk( x ) — Xk{v) = on the last subdomain. 
Moreover, we obviously have 



1 



s,s'aTi J0<r<3/k, 3/k<r><6 Vr' \\/r + Vr 7 ) \x - y\ 



;\Xk(x) ~ Xk(y) \ dxdy 



< 



0<r'<3/fe, 3/k<r<S Vr'i^ + Vr 1 ) \x - y\ 



:\Xk(x) - Xk(y)\ dxdy. 
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Therefore we focus on the estimates of the integral on the first two subdomains above. First, 
since Xk is a Lispchitz function with a Lipschitz constant of order k, we have 



/ / 

Js.s'er, J0<r,r' 



1 



er 4 Jo<r,r'<3/k Vr 1 '{y/r + y/r 7 ) \ x - y\ 



\Xk(x) -Xk(y) \ dxdy 



s,s'eri Jo<r,r'<3/k y r' {y/r + Vr') \x — y 

y/r 



0<r,r'<3/fc J{QM) 2 V?{y/¥ + V?) {{s - s') 2 + (r - r') 2 ) 1 / 2 

In I r — r I I dr dr 



< Ck 

< Ck 

< Ck 

Ck 



< Ckx^x^. 
y/k k 



1 



dx dy 
1 



ds ds' dr dr' 



0<r,r'<3/k vr'(y/r + Vr') 
3/fc dr / \ / /-3/fc 



In \z\ dz 



3/k 



As for the second subdomain, since Xk( x ) = 1 f° r r > 2/k, in fact, the domain of integration 
in r' is only r' < 2/k. Since Xk is bounded by 1, we have 



/ / 

Js.s'PFi J0<r'< 



1 



, , . ,- , , , j2\Xk(x) - X k{y)\ dxdy 

'eTi J0<r'<3/k, 3/k<r<8 Vr {y/r + Vr) F _ 2/1 



< c 

< c 

< c 

< C 



I I 

J0<r'<2/k, 3/k<r<8 J (0 



y/r 



o<r'<2/fe, 3/fc<r«5 J \om) 2 Vr 1 {y/r + Vr 7 ) {s - s') 2 + (r - r') 2 

dr dr 



(is ds' <ir dr' 



0<r'<2/fc, 3/fc<r<<5 vr 1 {y/r + Vr 7 ) |i" - 



2/k dr / 



5 

1/fc z 



In /c 



vV 

2. Proof of (3.19): we use the same type of domain decomposition as above; the only difference 
lies in the fact that r' may take negative values. 

• On the subdomain r > 3/k,r' > 3/k, the integral is identically zero; 

• On the subdomain 3/k < r < 5, —5 < r' < 3/k, we have Vxfc(^) = and as soon as 



r' > 2/k, 



Xk(x) - xk(y) - Vxfc(x) • (x - y)i\ x - y \<5/2 = 0. 
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Therefore 



jz/k<r<5,-5<r'<^k, ^T^3 Wk{x) - Xk (y) ~ V Xk (x) ■ (x - y)l\ x - y \< 5/2 \ dx dy 



s,s'er 



- C I I 77 A2 7~T J^zl2 ds ds ' dr dr ' 

J3/k<r<5,-5<r'<2/k J (OM) 2 \( s ~ s ) + ( r ~ r') 2 ) 6 / 2 



< C f ^ dr dr' 

J3/k<r<6-6<r><2/k \ r ~ r I 

< C t -^dr. 

Js/k r - E 

Using the inequality 



we infer eventually that the integral is bounded by C (^/S + ■ 
On the subdomain < r < 3//c, |r'| > 3/fe, we use the bound 

lxfc(«) - Xfc(y) - Vxfc(a:) ■ (s - y)i| x -i/i<*/2l < Ck \ x - y\, 

so that 



/0<r<3/fc,3/fc<|r'|<<5, \ x _ y|J 

• / V '', dr dr' 

'0<r<3/k,3/k<\r'\<5 \ r ~ r I 
r 3/fc 



Jo<r<3/k,3/k<\r'\<S, ] x ~^ \ Xk ^ ~ Xk ^ ~ VXk ^ ' ^ " ^l^l^l dx ^ 

/ 

rS/k / 3 X 

< — C7c / v^fo i — r <^ r 

< Ckx^-x 1 ^. 

Vk k 

• On the subdomain < r < 3/k, \r'\ < 3/k, we use a Taylor-Lagrange expansion for the 
function Xk, which yields 

\Xk(x) - Xk(y) ~ Vxfc(x) • (x - y)l\ x -y\<5/2\ < Ck 2 \x - y\ 2 , 

so that 

J0<r<3/k,y\<3/k, |^3 Uk(x) ~ X k(y) - VXk(x) ■ (X - y)l\ x - y \<6/2\ dx dy 

< Ck 2 I y/r | In \r — r'\ | dr dr' 

JQ<r<3/k,\r'\<3/k 

< ck 2 ^k-V 2 . 

k 
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3. Proof of (3.20): this term is easier to estimate than (3.18), (3.19). We merely have 

|Vx*(x)| 



TlxTl 



\x - y\ 



dx dy < C 



< C 



{-6,8)* 
28 



k\x'(kr)\ |ln|j — r'\ I dr dr' 



/AO 
|ln \z\ | dz 
-28 



< C5\lnS\. 



4. Proof of (3.21): since Xk(y) = if r' < 0, we have 



1 r>0,r'<0 
lr>0,r'<0 



x — y\ 3 
x-y\ 



\Xk(x) - Xk(y)\ dxdy 
gXfc(x) dxdy 



o r s 



< C 

< C 

C_ 

~ Vk' 



8 JO 

2/k 



— 2 x(kr) dr dr' 



o 



r r + 5 



dr 



□ 

We now address the rest of the proof of Theorem [TJ With the help of Lemma 15 the 
estimation of the remainder terms in is immediate. We set 



w (x) := l r>0 



^i(s) 



wi(x) := Jr^^is) + u 2 (x), 



so that iio = wq + wi. We write 

uo{y)ixk{x) - Xk(y)) - u {x)Vxk{x) ■ (x - y)i\ x - y \<8/2 
= u (x) [xk{x) - Xk(y) - ^Xk(x) ■ (x - y)l\x-y\<6/2] 
+(uo(y) - u (x))(xk(x) - Xk(y))- 

There exists a constant C such that 

\ui{x)uq{x)\ + \y/r^il)o{s)wi{x)\ < C^/r^l \fx G M 2 , 

so that 

ui(x)u (x) 



+ 



T s xT s \x~y\ 

y /r^ijj (s)w 1 (x 

T s xT 5 



[Xk{x) - Xk(y) - Vxfc(x) • (x - y)l\ x -v\<8/2] dx dy 



\x - y\" 



[Xk(x) - Xk{y) ~ VXfc(x) • (x - y)l|a;-j,|<5/2] dx dy 



< (3.19). 
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On the other hand, simple calculations show that 

\uo(x) - uo(y)\ < C\x-y\ ( I + 
+Cl rr '<o 



r,r'>0 



1 



\wi(x) - wi(y)\ < C\x-y\[l + 



lr,r'>0 



+Cl rr , <0 (v^+ + Vr 7 +) 



As a consequence, 

\ui{x)\\u Q (x) - u (y)\ < C\x - y\ + l r y 



r'(y/r + v r') 



+ Cl r> o, r '<OV^', 



so that 



/r^rVo(s)| \wi(x) - wi(y)\ < C\x - y\ ^/r^ + l rr / >0 



= j + Cl r> o,r'<0^ 



+ 



v w (x) - uo(2/))(XfcO) - Xfc(y)) ^ 

T s xT a \x-y\ 6 



T s xT s \X~y\ 



-(\wi(x) - wi(y))(xk(x) - Xk(y)) dx dy 



< (3.18) + (3.21). 



Gathering all the terms, we deduce that 



lim lim I k = C 2 } / ipo^i, 



where C2 is an explicit positive constant. Moreover, using formulas (3.6), (3.7), we have 



where ((x) = 4>o(x). Eventually, we deduce that 
dJ f (n t ) _ CiC 2 — rU 



dt \t=o 



J2 [ \d 1 J 2 u Q (s))\-n{s)ds 
i=i Jo 



C1C2 



on 



(d 1 J 2 u (s)) 2 C-n(s)da(s). 



Therefore Proposition [4] is proved, with Co := — {C\C-i)l^- 



3.5 Proofs of Lemma 12 and formulas (3.4), (3.5) 



• The proof of (3.3) follows closely the one of Theorem 5.3.2 in |lUj . The only differences come 
from the mixed boundary conditions and the fact that (fit only affects horizontal variables. 
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The key point is to prove that Vt is differentiable with respect to t with values in H 1 
To that end, observe that Vt is the solution of the elliptic problem 

-div (AtVVt) = in R\ , 

d z Vt = fo<j H aanx{0}, (3.23) 
V t = on Q, c x {0}, 

where 

•4 t = de,( W AW<)-W)-' 



1 



(Notice that V0t is a 2 x 2 matrix. 



Indeed, (3.23) is easily proved by writing the variational formulation associated with the 
equation on U% and performing changes of variables. Since the latter are strictly identical to 
the ones of [TU], we skip the proof. 

For further reference, we also write the system derived by lifting the Neumann boundary 
condition. We set V t (x,z) = V t (x,z) + f(4> t {x))9(4> t {x), z), where 9 G Co°(IR 3 ) is such that 
9(x, z) = z for x in a neighbourhood of and \z\ < 1. Then Vt solves 



-div (AtVVt) = | det(V&)|(A(/0))(&(s),*) on M 3 , 
d z V t = on n x {0}, V t = on Q c x {0}. 



(3.24) 



Let 

V := {V G £f oc (R^), G L 2 (M 3 ), l/ = 0onO c x {0}} . 

Then according to the Hardy inequality in M 3 , there exists a constant Ch such that for all 
V eV, V(l + \x\ 2 + \z\ 2 )- 1 / 2 G L 2 (M 3 f ) and 



7rs 1 + kl + pi 



This inequality is usually stated in the whole space, but a simple symmetry argument shows 
that it remains true in the half-space. Therefore || W||x 2 is a norm on the Hilbert space V. 

Now, for t in a neighbourhood of zero and V G V, define the linear form F(t, V) G V by 
VW G V, (W, F(t, V)) = [ (AtVV)-VW- [ \det(V<f) t )\(A(f9))((j)t(x,z))W(x,z)dxdz. 

JR3_ JR3 

Notice that F(t, V) = is the variational formulation associated with the equation (3.24). 
We then claim that the operator 

F: (t,V) GK xV^ F(t,V) G V 

has C 1 regularity for t small enough. Indeed, t ^ A t G L°°(R 2 ,M3) is C°° for t 
neighbourhood of zero. On the other hand, for (A, V) G L°°(R 2 , M 3 ) x V, W, let 

a(i4, F) : W G V i-> / (AVV)-VW- [ \det(y<pt)\(A(f9))(fo(x,z))W(x,z)dxdz. 
Jr% Jr 3 + 



m a 
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Then the application 

(A V) G L°° (R 2 , M 3 ) x V i-> a(A, F) € V, 

has C 1 regularity since it is the sum of a bilinear and continuous function and a constant term 
(with respect to A, V). 



Let V G V be the solution of (3.24) for t = 0, i.e 



-AVb = A(/0), 
a 2 Vb = on x {0}, Vb = on ft c x {0}. 

Now, for W G V, D v F(0, Vq)W is the linear form 



i.e. the scalar product on V. Therefore 

D v F(0,y o ) :V4V' 

is an isomorphism. The implicit function theorem implies that there exists a C 1 function 
1 1— > V(t) in a neighbourhood of zero such that 

F(t,V(t)) = 0. 



Uniqueness for equation (3.24) yields V% = V(t). We infer immediately that 1 1— )• Vt is C 1 in a 



neighbourhood of zero. We now use the following trace result: for any U G V, 

||t7| 2= o|| H i/2( R2) < CUVf/ll^^). (3.25) 

Indeed, it is well known that if U G V, the following estimates hold 

ll^l^ollfl-i/a^a) < CHVi/H^RS), 
||^U=o||^( K2 ) < C||VC7|| i2(K 3 j. 

There only remains to prove that the I? norm of C/| z= o is bounded by ||VC/||^2. Since 
U\ z =o = on f2 c , we have 

||^|z=oIIl»(r») < N 1/2 II^L=olb 

< C\\VU\\ L z {R3 y 



Therefore (3.25) is proved. As a consequence, t^v t G F^QR 2 ) is C 1 in a neighbourhood of 



zero. The result on ut follows almost immediately by differentiating the formula 

u t = v t o c/^ 1 

with respect to i. The only difficulty comes from the fact that Vt is a priori not smooth 
enough with respect to x in order to use the chain rule. However, we can write 



u t -u = (v t -v )o<p t 1 + (yo o (j) t 1 - vq) 
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Since v t G C([-5,5\, H l / 2 {R 2 )) for 8 > small enough, the first term is 0(t) in L 2 (R 2 ). As 
for the second one, if vq were smooth, say vq G ff 1 (IR 2 ), then we could write 



d 
dt 



It can be easily checked that the right-hand side is bounded in H 1 ' 2 by C||«o||fl-i/a, where 
C is a constant depending only on (fit- Therefore if v$ G H , t > 0, 

IN ° (Pt 1 ~ V \\ H -i/2^ < Ct\\v Q \\ H i/2(jgay 

By density, this inequality remains true for all vq G H 1 / 2 . We conclude that itt G H^ 1 / 2 . 



We end this section by proving the asymptotic expansions (3.4), (3.5). We apply the results 
of Chapter C in [7] to the function Vt ■ We start by extending Vt on R 3 \ x {0} by setting 

V t (x, z) = -V t (x, -z) x G R 2 , z < 0. 

Without any loss of generality, we assume that the function 6 G Cq°(IR 3 ) defined in the proof 
of Lemma 12 is odd with respect to z. 

Then the extended function Vt satisfies 

V t = Vt + f(Mx)MMx),z), 
-div (A t VVt) = | det(V&)|A(/0)(&(a;),z) on R 3 \ Q x {0} 

<9^ = on O x {0}. 

For every connected component Tj of d£l, we introduce a truncation function G C^°(IR 2 ) 
such that 

tpi = 1 on a neighbourhood of Tj, 

Supp </>i n (an \ r<) = 0. 

Since the support of Vy?j is separated from the zones where Vt has singularities, we infer that 
for all t, (fi(x)Vt(x, z) solves a boundary value problem which is elliptic of order two in the 
sense of Agmon, Doughs, Nirenberg (see PQ, [2] and the definitions in chapter 7 of [8]), with 
homogeneous boundary conditions of order one and with C°° right-hand side. Moreover, we 
have proved in Lemma 12 that Vt G Hi oc (R 3 \ O x {0}). Therefore we can apply Corollary 
C.6.5 in [7J. As in paragraph 2.2 we denote by (r, 0) polar coordinates in the planes normal 
to r, and centered on Tj, and by s the arc- length on Tj, so that 

R 3 \ (Supp ipiDO, x {0}) = W n = {(s,r,6), s G (0,L f ), r > 0, 6> G (-vr,vr)}. 

We deduce that for all i, there exist functions ip®(t;s,6), tpj(t; s,9), which are smooth with 
respect to s, G [— tt, it], such that 

iPi{x)V t (x, z) = $(t; s, By 1 / 2 + $(t; s, Oyi 2 + u reg (t; x, z) + u rem (t; x, z), (3.26) 

with Uregit;-) G C°°(R 3 ) for all t and u rem (t;-) G C 2 (W^) with d^u rem (t; x, z) = o(r 3 / 2 -^\) 
for all t and for any multi- index (3. Setting 

Vl(t;x,z) := ^(i;s,0)r 3/2 + u reg (t;x,z) + u rem (t; x , z) + 9(<p t (x), z), 
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we have G C X {W^), V±(t\ x, z) = 0{r) for all t, and 

V t {x,z)=$°(t;s, e)r 1 ' 2 + V- L {t;x,z) 
for x in a neighbourhood of IV Since 

u (x) = lim V (x,z), 

z^0,z>0 



we obtain decomposition (3.4) with 



ifj°(s) := "0^(0, s, 7r) for s G ri,ui(ic) = lim Va (0;x, z) for x G Supp^i 



Differentiating (3.5) requires regularity results with respect to i on the terms of the decom- 



position (3.26). Using for instance Theorem C.6.2 in [7], or looking precisely at the details 
of the proof in section C, it is easily proved that the functions ip°, ip 1 , u reg and u rem are 
differentiable with respect to t, and that their i-derivatives are smooth with respect to x. 
Denoting by s(x), r(x) the local coordinates of a point i£R 2 , we have 



so that 



u t (x) = $°(t, s(^\x)), 7r)(r(^- 1 (x))) 1 /2 + V \t, ft\x), 0+) 



2 r{(p t {x)) 1 / 2 

+8^ (t, ^ l (x) , 0+) + d t {fo\x) ) ■ W 1 (t, ^\x) , 0+) . 

l 



(3.27) 



Therefore ut G Lf 3 (L^,), which proves (3.8). We recall that 4>o = (f> Q 
We infer 



Id, and that C = No- 



where 



2y^ 

= (^° + C-T^°) (0, S) 7T), 

U2 {x) = dtV 1 ^, x, 0+) + C • VV\0, x, 0+). 



Thus (3.5) is proved. 



4 Radial symmetry by the moving plane method 

The aim of this section is to prove Theorem [2} This theorem extends the well-known theorem 
of Serrin [14] on the classical laplacian, for which (1.5) is replaced by 



f -An = 1, x G 

u = 0, x G <9f2, 
d n u = C, x G <9f2 



(4.1) 



The proof of Serrin uses the celebrated moving plane method, and we will adapt it to our 
fractional setting. 
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Figure 1: Configuration 1 (on the left) and configuration 2 (on the right). 



4.1 Reminders on the moving plane method 

We remind here the main arguments of Serrin's proof. The starting point is the introduction 
of a family of hyperplanes (lines in our 2d case), say H\ := {x 6 M 2 ,xi = A}, parametrized 
by A € M. We also define, for any function u defined on M 2 , R\u{x\,X2) '■= u(2X — x\,X2) 
the reflection of u with respect to H\. For A small enough, H\ does not intersect the domain 
0. Increasing A, that is moving H\ from left to right, one reaches a first contact position, 
corresponding to 

A := inf{A,# A nfi ± 0}. 

Up to a translation, we can always assume that Ao = 0. Thus, for A > 0, we can consider the 
cap Yj\ := n n {x% < A} and its reflection S' A with respect to H\. Note that, for A > small 
enough, is non-empty and included in O. As A increases, it remains included in at least 
until one of the following two geometric configurations is reached (see Figure [TJ: 

1. is internally tangent to the boundary of Q at some point P not on H\. 

2. H\ is orthogonal to the boundary of O at some point Q. 

Let A be the first value of A for which configuration 1 or 2 holds. Note that we may have 
Ea C £1 for some A > A, but this will be irrelevant in the proof. The main point in Serrin's 



proof is to show that one has R^u = u inside where u is the solution of (4.1). This 
fact yields easily the symmetry of Q, with respect to H^. As the direction of H\ was chosen 
arbitrarily, it will folllow that for any direction, there is an axis of symmetry for £1 with that 
direction. This property implies that 0, is a disk. 
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To show the identity R^u = u inside Sa, one argues by contradiction through three main 
steps. Assume that the equality does not hold. Then 



Step 1. One shows that the function w := u — R\u is positive inside E A . 

Step 2. One obtains an upper bound for w near the tangency point P (configuration 1) or Q 
(configuration 2). As the normal derivative of u is constant along dCl, one has w = d n w = 
at P or Q. For configuration 2, one can show furthermore that the second derivatives cancel 
at Q: dijiu = for all i,j = 1,2. Denoting by r the distance at the tangency point, these 
properties imply that 

w = 0(r 2 ) at P (configuration 1), or w = 0(r 3 ) at Q (configuration 2). (4.2) 



Step 3. One shows a lower bound which contradicts (4.2). For configuration 1, this lower 
bound is obtained easily. Indeed, one knows from Step 1 that w > inside S A . Hence, Hopf's 
lemma implies d n w > at P (where n is the inward normal), which shows that w should 
grow at least linearly with r inside the domain. For configuration 2, S A is not regular enough 
at Q to apply Hopf's lemma. However, one can still show that 

d s w > or dgW > at Q 

for any direction s entering S A non-tangentially: see |14^ Lemma 1]. It follows that w should 
grow at least like r 2 inside the domain. In both cases, we reach the targeted contradiction. 
We refer to 14 for all details. 



Our ambition is to transpose this scheme of proof to problem (1.5). Due to the nonlocal 
character of (—A) 1 / 2 , it requires many modifications: 

In the case of the Laplacian, Step 1 follows from a simple application of the strong maximum 
principle, as w > on In our fractional (and therefore nonlocal) setting, such use of the 

maximum principle is impossible: it would require that w > on the whole M 2 \ £», which 
is not true (w is an odd function) . The appropriate treatment of Step 1 will be addressed in 
paragraph |4.2[ 



In the case of the Laplacian, Steps 2 and 3 rely on the regularity of u up to the boundary. In 
the case of (—A) 1 / 2 , we only have C°' 1//2 regularity of u, which implies substantial changes. 
Loosely: 



the upper bounds will follow from the asymptotic expansion ( 2.8 ) of it near the boundary. 



— the lower bounds will follow from the construction of refined subsolutions. 



Additionally, we emphasize that the constant c in (1.5) is necessarily strictly positive. 
Indeed, c > by the maximum principle. The fact that c > is a consequence of the Hopf 
Lemma for the fractional laplacian. In the present context, we may present a self-contained 



proof: if c = 0, we jump to paragraph 4.3 The arguments developed there (for the function w 
instead of u) allow us to conclude that u = 0. This is excluded by the equation (— A) 1 / 2 ^ = 0. 
Therefore we restrict our analysis to c > 0. 
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4.2 Positivity of w 

We shall first prove that if non-identically 0, w := R\u — u is positive inside Sa (which 
amounts to achieving Step 1). We shall rely on ideas developed by M. Birkner, J. Lopez- 
Mimbela and A. Wakolbinger in article [3|. They show there the radial symmetry of solutions 
of some semilinear fractional problems 

(-A) a u + F(u) = 0, xe 5(0,1), u = 0el n \5(0,l) 

set in the unit ball B(0, 1) of M n . Their proof is based on an adaptation of the moving 
plane method, and its baseline can be used to show the positivity of w. Nevertheless, several 
changes are needed, and simplifications of the original arguments can be made, as we now 
describe. 

First of all, we introduce the set 
£ ■= {Ae]0,A[, s.t. V0 < 7 < A, RyU-u>0 on S 7 , and diu > on H 7 nU}. 
The main point is to show that C =]0, A[. We proceed in several steps 

• Statement 1: ]0, e] C C for e > small enough. 

To prove that ]0,e[c £, it is enough to show that d\u > for all x G f2 with < x\ < e. 

1 /2 A 1 

From the condition d n u = c > 0, and the expansion (2.7), we know that 

d r u = \cr- 1/2 + 0(1), d s u = 0(1) 

as the distance r to the boundary (measured inside the domain) goes to zero. It follows that 

d\u = d\r d r u + disd s u = (d\r) r~ 1//2 + O(l), r — > 0. 

Moreover, for < A < A, for all inward normal vectors n along dVl n {x\ < A}, one has 
ei • n > 0, where e\ = (1,0). It follows that d\r has a positive lower bound for < x\ < e 
small enough. The statement follows. 

• Statement 2: C is an open subset o/]0,A[. 

The point is to show that if A G £, [A, A + e[c C for e > small enough. This statement will 
be deduced from the following: 

Lemma 16. Let A < A. Then 

i) For all x G dCl D {xi < A}, R\u(x) - u(x) > 0. 

ii) Assume that R\u — u > on Y>\. Then, R\u — u > on In particular, one has 
R\u — u > on T,\ for any A G C 

Proof. We remind that u > in fl by the maximum principle. 

i) As A < A, for all x = {x\, X2) G d£l n {x\ < A}, x\ := (2A — x\, X2) G Q.. In particular: 

R\u(x) — u(x) = u(x\) > 0. 
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ii) We assume that w := R\u — u > on E>. Note that on {x\ < A} \ Y,\, one has 
w = R\u > 0, so that w > on the whole half-space {x\ < A}. Note also that by i) and the 
continuity of u, the fonction w is not identically zero in We want to show that w > in 
A- 



We introduce the harmonic extension W of w, defined on 1R5_. We remind that 



AW = on R%, W\ z=0 = w on M 2 . 

Moreover, as (— A) 1 / 2 ?^ = 1 in O, c^M<^| 2= o = in S^. We can express W with the Poisson 
kernel: 

. z /" dt\dt 2 . , 

W(xi,x 2 ,z) = ^- - 2 2 u>(ti,t 2 ). 

2tt 7 R 2 ((xi - ti) z + [x2 - t 2 y + z i yi i 



As R\w = —w, this integral formula can be written 
W(xi,x 2 ,z) = — / dtidi 2 

1 



((XI - tl) 2 + (X 2 " t 2 ) 2 + Z*y/ 2 (Xi + (h - 2A)) 2 + (X 2 - t 2 ) 2 + Z 2 )V2 



w(ti,t 2 ). 



As it) > on {ti < A} and not identically zero, we deduce from this formula that W > for 
x\ < A, z > 0. 

To conclude on the positivity of w, we assume a contrario that w(x*) = for some 
x* = (xi,x 2 ) 6 £>. The function W is smooth up to the boundary in the vicinity of 
(x*,X2,0), and = W(x*,xl,Q) < W(x,z) for all (x,z) eK 3 + . By Hopf's lemma, we should 
have d z W{x\, x 2 , 0) > hence reaching a contradiction. This concludes the proof. 

□ 

Back to Statement 2: we argue by contradiction, as in [3l paragraph 4.2]. We assume that 
there is A € £ and a decreasing sequence (A n ) c]0, A[\£ converging to A. Up to extract a 
subsequence, we can assume that 

1. either there exists a sequence (x n ), x n £ Y,\ n for all n, such that x n — > x* and u(x n ) > 
R\ n u(x n ) for all n. 

2. or there exists a sequence (x n ), x n G H\ n n $7 for all n, such that x n — > x* and c9in(x n ) < 
for all n. 

We first consider case 1: since A 6 £, we have — u > on As u is continuous, we 



get u(x*) — R\u(x*) > 0. By Lemma 16 we cannot have x* S T,\ U (c9f2 n {xi < A}). Hence, 

— 1 /2 

x* £ H x n f2. If x* G H\ n one has 5 n n(x*) = c > 0. Moreover, as A < A, &\ ■ n* > for 
n* the inward normal vector at x*: indeed, since A < A, configuration 2 has not been met, 
and therefore e\ ■ n{y) does not vanish for all y G d£l such that y\ < A. Since e\ ■ n(y) = 1 
when y = (0, y 2 ) G dU, we infer e± ■ n* > 0. 

Reasoning as in the proof of Statement 1, we obtain that u is strictly increasing with xi 
in the vicinity of x* (inside the domain). Hence, for n large enough, we get 

R\ n u(x n ) - u(x n ) > 

in contradiction with the assumption u(x n ) > R\ n u(x n ) for all n. If x* G H\ n fi, then 
d\u(x*) > (because A G £). This means again that u is strictly increasing with x\ in the 
vicinity of x*, which yields the same contradiction as before. 
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It remains to consider case 2. With the same reasoning as in case 1, we get that x* £ 
H\ n 0, and that u is strictly increasing with x\ in the vicinity of x* . This contradicts the 
assumption d\u(x n ) < for all n. 

Statement 3: C =]0, A[. 

Prom the previous statements on C, we know that C =]0, A max [ for some A max — A. Again, 
we shall argue by contradiction and assume that A max < A. For all A < A max , one has 
R\u — u > in Sa, so that by continuity of u, R\ max u — u > in ^A max - Lemma 16 implies 
in turn that 

RA max u-u> in ^\ max - 

We want to show that d\u > on H\ max n Q. This would imply that A max £ £, so the 
contradiction. 

As w := Rh max u — u is odd, it is enough to prove that d\w < 0. We introduce its harmonic 
extension, as in the proof of Lemma fl6j it satisfies 



AW = on R%, W\ z=0 = w on R 2 , d z W\ z=0 = on Z Amax . 

We extend W to the lower half-space by setting 

W(xi,X2,z) := W(xi,X2,—z) for z < 0. 

As d z W\ z= o = on SA maa . X {0}, this extension is harmonic through ^h max , that is outside 
(R 2 \£A mM; ) x {0}. Let G H\ max f]Q. W is smooth and harmonic near (x*, x%, 0). Moreover, 
one has 

= u}(x*) = W(xl,xl,0) < W(x,z), say for all (x, z) £ O := T, Amax x R 

Applying Hopf's lemma in O (for which (x\,X2,0) is a boundary point), we obtain that 
d\W{x\, X2, 0) < that is d\w{x*) < 0, as expected. 

As a conclusion of our analysis, we obtain that R\u — u > on for all A < A. 
By continuity of u, we obtain that w := R\u — u > on T,\. From there, there are two 
possibilities: 

• either w = on "E\. It follows easily that O is symmetric with respect to H\. As explained 
in paragraph 4.1, the direction of Ha being arbitrary, it follows that f2 is a disk. 

• or w(x) > for some point x in T,\. But then, following exactly the proof of point ii) in 
Lemma 16, we obtain that w > on Sa (or w := u — R\u > on 

The rest of the section aims at excluding the second possibility. From now on, we assume 
that w > on T,' A , and will establish contradictory lower and upper bounds on w. 

First of all, we state a lemma, to be used in both configurations 1 and 2. 

Lemma 17. Let W be the harmonic extension of w to M+. For all R > sufficiently large 
there exists Cr > such that 

W(x, z) > c R {x 1 - A) z, for all A < x 1 < R, -R<x 2 <R, < z < R. 



33 



Proof. Let R > large, and Q R :=]A, R[x] - R, i?[x]0, As W and W" := x (a* - A) z 
are both harmonic functions on we can use the maximum principle to compare them: 
we just have to show that W > crW' on 8Qr for cr > 0. We take R large enough so that 
the vertical sides of the cube (other than the one supported by {x\ = A}) do not intersect 
f2 x {0}. Then, we distinguish between the different sides: 

On 8Qr n {z = 0} or 8Qr n {xi = A}, one has W > (remind that w > on {xi > A}), 
and W' = 0. The inequality is clear. 

Let Fi^r := 8Qr n {xi = i?}. Let e > 0. As is continous and > on the compact set 
Tl r D{z > e}, one can find c = c e > such that > c W' . To have the same inequality on 
the whole Ti^r, it is enough to show that 

.. . . W(x,z) 1 ,. . , W(x,z) n oN 

limmf — \ = liminf — > q f^g\ 

(i,2)er 1>fl , W'(x, z) R — A (x,z)er ltR , z 

z^0+ Z-5-0+ 

This is a consequence of Hopf's lemma: W is positive harmonic on Qr D {z > 0}, satisfies 
W = on Ft 7 R n{z = 0}. It follows that d z W > on n {z = 0}. Note that <9 2 W exists 
thanks to our choice of R, away from f2. It is furthermore continuous, so that it is bounded 
from below by a positive constant. Inequality (4.3) follows. 

On r3 5 # := 8Qr n {z = R}, one can proceed exactly as in the case of Ti^: use the positivity 
of W away from {x\ = A}, and use Hopf's lemma to get d\W > c > at T^r fl {xi = A}. 

We now turn to I^ij = OQr n {x2 = ±R}. Away from the edge x\ = A, z = 0, we can as 
before use the positivity of W and Hopf's lemma to obtain W > cW . It then remains to 
handle the vicinity of X = (A, ±R, 0). The main point is to show that 

liminf W ^ X ^} > 0. (4.4) 
(x,z)er 3 , R (xi - A) z 

(x,z)-*X 

We can not apply Hopf's lemma: W is harmonic in the dihedra {x\ > A, z > 0}, but it does 
not satisfy the interior sphere condition at X. As W(x,z) = both for x\ = A and z = 0, 
one has 

J <9^(x, z) = for xi = A, \/k e N 
\ ^^(x, z) = for z = 0, V/c G N. 
((x, z) in the vicinity of X). Hence, to prove (|4.4[), it is enough to show that 



did z W(x, z) > c> 0, (x, z) G B(X, 5) n {x x > A, z > 0}, 

for small enough 5. By continuity of d\d z W . it is enough to show that d\d z W{X) > 0. This 
positivity condition follows straightforwardly from a famous lemma of Serrin, see [14[ Lemma 
1, p308]. In our context, it reads: for any vector s entering the region {x\ > A, z > 0}, we 
have d s W(X) > or d 2 s W{X) > 0. Taking s = e± + e 3 , we obtain that 

(3i + d z )W(X) > 0, or (9i + d z ) 2 W{X) > 0. 



Combining this statement with (4.5), we see that the first condition is not realized, and that 
the second one amounts to d\d z W{X) > as expected. 

□ 



34 



4.3 Contradictory bounds on w: configuration 1 
Lower bound 

We start with configuration 1, that is when is internally tangent to <9f2 at a point P not in 
H\. We introduce a small open disk Dr C of radius R, also tangent to 8ft at P. We take 
R small enough so that 8Dr n 8Q = {P}. Let xr be the center of Dr, and let (f> G [0, 2-7r] 
parametrizing 8Dr. Finally, for any x in Z)r, denote by p = /?(x) the distance between x and 
the circle 8Dr. One has x = xr + ((R — p) cos </>, (it! — p) sin <^>). The aim of this paragraph is 
to prove the following 

Proposition 18. Let /3 > 1/2. There exists cp > 0, pp > so that for all x G Dr with 
< p(x) < pp, 

w(x) > Cp p 13 . 

First, we shall extend the 2d coordinate system {p, 4>) to a 3d coordinate system (p, 
in the .R/2-neighborhood of Cr := 8Dr x {0} (see Figure [2]). Any (x,z) = (xi,X2,z) in this 
neighborhood has a unique projection p(x, z) on Cr, which reads: 

p(x, z) = (xr, 0) + (i?cos 0, R s'm<f>, 0). 

We introduce p = \ (x,z) — p(x,z)\ the distance between (x,z) and Cr, and G [— tt, ir[ the 
oriented angle between the vector (xr, 0) — p(x, z) and (x, z) — p(x, z). Then, one can write 

(x, z) = p(x, z) + {—p cos 9 cos (f), —p sin sin (f>, p sin 0) 

= (xr, 0) + ((i? — pcos 0) cos (i? — psin#) sin 0, psin^) 

The triplet (p,6,4>) defines a system of orthogonal curvilinear coordinates in the R/2 neigh- 
borhood of Cr. For 9 = 0, it matches the 2d coordinates (p,4>) of the previous paragraph. 



To prove Proposition 18, we shall rely again on the harmonic extension W of w. It satisfies 

AW = on R\, W\ z=0 = w on R 2 , 8 Z W\ Z=0 = on £' A . 

We can extend W to M 3 into an even function of z (still denoted W). Thanks to the last 
condition, it is harmonic through T,'^. We will show that 

W(x, z) > cp p 13 cos(#/2), \/p = p(x) < 5 small enough. (4.6) 



This will yield Proposition 18 for 9 = 0. This lower bound on W will follow from the maximum 
principle. We introduce Us := {x, p(x) < 5} \ {9 = tt}. The open set Us is obtained by 
removing from the ^-neighborhood of Cr the part of the plane {z = 0} outside Cr. Let Wp 
given in (p,9,4>) coordinates by Wp(x,z) = p^cos(9/2). We will show that for 5 >0 small 
enough, one has: 

i) AWp > (= AW) in U s . 

ii) W > cWp on 8Us, for some c > 0. 
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Figure 2: The coordinates (p, 9, < 



The lower bound (4.6) will follow 



Proof of i). One can check that the Laplacian in coordinates (p, 9, (p) reads 
A/ = 

i ( d ( a \ a ( R-pcosO a \ a ( P a 
' p{R-pcosd)—f\ + — [ — — / + — - — — -— / 



p(R — p cos 9) \dp \ dp J 89 \ p 89 J d(j> \R — p cos 

(4.7) 

This implies that 

AWp = _, n \___ n ^ (((P 2 ~ l/A)pP- x (R-pcosO)-PpP<xx6) cos(fl/2) - -pP sin#sin(#/2) 



1 


p(R 


— p cos 9) 




1 


p(R 


— p cos 9) 




1 


p(R 


— p cos 9) 



2' 

(/3 2 - l/A)p l3 - 1 (R -pcos9) - /?/ cos 9- /sin 2 (#/2)) cos(#/2) 
(P 2 - 1/4) Rpl 3 - 1 + 0(/)) cos(0/2) > 

for 5 (and p < 5) small enough. Note that we used the identity sin# = 2sin(#/2) cos(#/2) in 
the second line. 

Proof of ii). We must distinguish between different zones. 
On 8Us n {9 = ±7r}, one has W > 0, Wp = 0, so that the inequality is clear. 
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The compact set dU$C\{0 = 0} is included in E^, so that W > there, and as W is continuous, 
it is even bounded from below by a positive constant. In particular, one can find c > such 
that W > cWp. 

For any e > 0, we also know that W > on the compact set dUs D{\z\ > e}. We obtain again 
that W > cW/3 for some c = c e > 0. 

It remains to show that for e > small enough, one has 

W > cW p on dU s n {0 < |*| < e}. 

Due to the fact that W is non- negative in M 3 and even in the variable z, it is enough to show 
that 

liminf \W(x, z)/Wp{x, z)\ > 0. 

(x,z)edUs, 

To show such a property, we argue by contradiction. Assume that we can find a sequence 
(x k , z k ) satisfying: 



(x k ,z k ) C R%ndU s , z K ^0 + , W(x K ,z K )/W t )(x K ,z K ) 



„k „fe\ /TIT" /_& ^,fc^ 



0. 



Up to the extraction of a subsequence, we can assume that x k ,z k — > X for some X 6 (9L^ 
with X = (Xi,X2,0). There are two possibilities: 

— either W(X) > 0. As Wp is bounded, this yields easily a contradiction. 

— or W(X) = 0. In particular, X is outside Cr, which implies in turn Wp{X) = 0. More 
generally, W(xi,X2,0) = for all (xi,X2) close to (Xi,X2). Moreover, as Wp is smooth 
near X, we obtain 

|Wjg(a;, z)\ = O(z), as z — > + , i in the vicinity of X. 

From there, we get that 

limsup \W{x k , z k )\/z k = (4.8) 

k— >co 



As x\ remains away from A, this limit contradicts Lemma 
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Upper bound 

We now turn to an upper bound for w. Let x £ T,' A . For x close to P, we can write in a 
unique way 

x = x* + rn, resp. x = x* + f re, r, f > 0, 

where G 30, resp. x* £ and re = n(x*), resp. n = n(x*), refers to the inward normal 

to dO, at x , resp. to the inward normal to dT,' A at x. We now use the results of Costabel et 
al [7], and the fact that d n u = c at the boundary. Following (2.8), we get the expansions 



u(x) = cr 1/2 + 0(r), R A u(x) = cr 1/2 + 0(r), r, r -)■ 0. 

Now, we remark that the inward normal vectors n and n coincide at P. The expansions above 
then lead to the upper bound 

w(P + rn) = 0(r), r > small enough. (4-9) 
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Furthermore, the coordinate r along the normal at P coincides with the coordinate p from 



Proposition 18 Applying this proposition with /3 = 3/4, we get 

w(P + rn) > Cr 3 ^, r > small enough 



Comparison between this lower bound and the upper bound (4.9) gives a contradiction. 



4.4 Contradictory bounds on to: configuration 2 

In this paragraph, we investigate configuration 2, in which H\ is orthogonal to d£l at some 
point Q. 

Lower bound 

We shall prove the following 

Proposition 19. Let v = (1,^) he a vector entering at Q. Let f3 > 1/2. There exists 
C/3 > such that 

w(Q + tv) > cpt 1+ P , for t small enough. 
To prove this proposition, we consider again a small disk Dr, of radius R, tangent to d£l 



at Q. Exactly as in the proof of Proposition 18, we introduce the 3d coordinates (p,9,4>) in 



a i?/2-neighborhood of Cr = 8Dr x {0}. We also introduce the 3d harmonic extension W 
of w, even with respect to the variable z. Our aim is to prove a lower bound on W which 



implies Proposition 19, namely 



W(x,z) > cppP cos (0/2) COS0, V — 7r/2 < (j) < 7r/2, V0 € [— 7T,7r], \/p = p(x)<pf3 

(4.10) 

small enough. Therefore, let Wp(x, z) = p 13 cos(#/2) cos 4>. Comparison between W and Wp 
will come from the maximum principle. The main change with respect to configuration 1 is 
that W = on the hyperplane {x = (xi,X2,z), x\ = A} (corresponding to 4> = ±tt/2). We 
have to restrict to the open set 

Uj = U s n {x! > A} = {p < 5, xi > A, 9 / it}. 

We shall prove that 

a) AWp > (= AW) on U+. 

b) W > cpWp on dUj, 5 > small. 



Proof of a). We use again formula (4.7). It gives 



cos(0/2)cos( 



R — p cos 9 
1 

p(R — pcos 9 



(/3 2 - 1/4) Rp 15 ' 1 + 0(/)) cos(0/2)cos(/. > 0. 
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Proof of b). Let e > 0. Away from H\, that is over dU$~ n{xi > A + e}, we can perform 
exactly the same analysis as in the proof of Proposition |18[ point ii). Thus, 



W > cWp, over dU£ n {x\ > A + e} for some c = c e > 0. 

It remains to treat the vicinity of H\ inside dUf . For x\ = A (that is <f> = ±7r/2), one has 
W(x, z) = Wp(x,z) = 0. Also, for 6 = ±n, one has W(x,z) > 0,Wp(x, z) = 0. Everywhere 
else, Wp(x, z) > and W(x, z) > 0. Hence, it remains to show that 

liminf \W(x, z)/Wp(x, z)\ > 0. 

(x,z)GdU+\{\e\=ir}, 

Again, we argue by contradiction: we assume that there is a sequence 

(x k ,Z k ) CdU+\{9 = TT}, xf->A+ ty(x fc ,z fc )/^(x fc ,z fe ) ->o. 

Up to a subsequence, we can assume that (x k , z k ) — > X for some X = (A, X2, Z) G dUf . As 
and Wy3 are even in 2, we can also assume that z k > for all fc. There are several cases: 

(X\,X2) = (A, X2) € f2. is harmonic and positive on {(x, z), | (x, z) — X\ < e,xi > A}, 
e > small enough. Moreover, for all (x,z) in this set, we have W(x,z) > = W(A,x%,z). 
We can apply Hopf's lemma, which yields d\W(A, X2, z) > C > 0. Hence, 

r W(x k ,z k ) 
liminf , \ - , / > 



k— >oo 



A| 



which yields in turn 



lim inf 



W(x k , z k 



> 0. 



fe^-oo |W^(a; fc ,z fe )| 

Indeed, | W^(x fc , | = \Wp(x k , z k ) - Wp(A, x k , z k )\ = 0(\x\ - A\) due to the regularity of 
Wp near X. Thus, we reach a contradiction. 

(Xi, X2) = (A, X2) ^ fi, 2 / 0. We are still in a situation where we can apply Hopf's lemma 
to W near X, which yields the same contradiction as above. 

(Xi,A2) = (A, X2) ^ Z = 0. Although W is harmonic and positive on the dihedra 
{x\ > A, z > 0}, we can not apply Hopf's lemma because the dihedra does not satisfy the 
interior sphere condition at X. Nevertheless, thanks to Lemma 17 we get that for some 
c > 0, for all k large enough 

W(x k ,z k ) > c(x k -A)z k > c' (\-\<t> k \) (K-0 k ) > c"Wp(x k ,z k ). 



This ends the proof of (4.10), and so the proof of Proposition 19 
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Upper bound 

We now look for an upper bound for w along the ray Q + tu, V = (l,u), t > small. We 
show 

w(Q + tu) = 0(t 2 ), as t goes to 0+. (4.11) 



Of course, such an upper bound leads to a contradiction with Proposition 19 (for 1/2 < /3 < 1). 

As in configuration 1, this bound follows from the behaviour of u near dfl, as given by 
(2.8). We remind that for x near Q, 



u(x) = C r 1/2 + Ci(xi)r 3/2 + u reg (x) 

where 

r = r(x) is a normal coordinate in the vicinity of d£l (which corresponds to r = 0). 
Co is the constant fractional normal derivative at dQ, C\ is a smooth function of x\. 



u reg is C near dfl, and vanishes at 90. 



We can then write 

w(x) = C w l/2 (x) + Ciw 3 / 2 (x) + w reg {x) + 0(\xi - A|r 3/2 ) 
with d := Ci(A), 

Wi/ 2 (x) := r 1/2 (x) - r 1/2 (x), w 3/2 (x) := r 3/2 (x) - r 3/2 (x), w reg (x) := u reg (x) - u reg (x), 

where x := (2A — xi,x 2 ). Note that Wx/ 2 , w 3 i 2 , w re g vanish at Q. 

Let us first prove that \7w reg {Q) = 0. The idea is the same as in the paper of Serrin |14j . 
Clearly, w re g(A, •) = 0, so that d 2 w re g(A, •) = 0. In particular, d 2 w reg (Q) = 0. We then have 
to show that d\w reg {Q). Near Q, we can write 90 as a graph: x 2 = ip(x\), with ip'(A) = 0. 
Moreover, 

U reg (xi,lfj(xi)) = diU reg + lp'd 2 U reg = 0, 

so that diu reg (A, 0) = 0, and from there, d\w reg {Q) = 2d\u reg {Q) = 0. It follows that 

Wreg{Q + tu) = 0(t 2 ) for t ->■ 0+. 

We still have to control w a , a = 1/2 or 3/2. For x close enough to 90, we denote p = p(x) 
its orthogonal projection on 90. Near Q, it reads p = (pi,ip(pi)), with 



il-pi W 1 
x 2 -ip(pi)J W(pi) 



0. (4.12) 



Then, r(x) 2 = (x\ - pi) 2 + (x 2 - ip(pi)) 2 = (1 + V'(Pi) 2 )(^2 - ip{pi)) 2 , and 

tWaCx) = (1 + ^( P i) 2 ) a / 2 |X2 " IpipiW " (1 + ^(Pl) 2 ) a/ V2 " ^(PlT 

with p := p(x). One may then Taylor expand w a (Q + tu) with respect to t (x\ = A + t, 
x 2 = ip(A) + ut). Indeed, relation \A.12\ : 

A + t-pi + (V>(A) + i/f - V(Pl) W(Pi) = 0, 
resp. 2A - (A + 1) - pi + (-0(A) + i/i - = 
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leads to pi = A + t + 0(t 2 ), resp. pi = A - t + 0(t 2 ). Then, 

|x 2 -V(pi)r = |^| Q + 0(* 1+a ), resp. \x 2 -HPi)\ a = \vt\ a + 0(t 1+a ). 

and 

(1 + V'(R) 2 ) Q/2 = 1 + f ^(A)t 2 + 0(t 3 ), resp. (1 + V'(pi) 2 ) a/2 = 1 + |<(A)t 2 + 0(t 3 ). 

Combining previous Taylor expansions yields: w a (Q + tv) = 0(t 3+a ) as t -)■ 0+. Together 
with the estimate w reg (Q + iz?) = 0(i 2 ), it implies the upper bound ( |4.11 ). 

Remark 20. If i/ie domain Q is not connected, our method can still be applied, provided the 
connected components of £1 can be "ordered", in the following sense: for every direction of the 
moving hyperplanes H\, for all A < A, H\ intersects at most one connected component o/fL 
In this case, it can be checked that our arguments remain valid. In particular, if $7 consists 
of concentric rings, we infer that 0, is a disc. 



Appendix 



This appendix is devoted to the calculation of the jacobian of the change of variables x = 
(xi,X2) — > (s,r), where x belongs to a tubular neighbourhood T$ of s is the arc-length, 
and r = d(x, <9r2)(21 xg n — 1). Using the same notation as in Section [5J we write 

x = p(s) — rn(s), 

where p(s) S M 2 is the point of dCl with arc-length s, and n(s) is the outward pointing normal. 
We deduce that 



Ox 
ds 



t(s) — r 



dn 



ds 



(1 + n{s)r)T(s), 
3x 

Tr = " n(s) - 

Let 9 be the oriented angle between e\ and n. Then, up to a change of the orientation of the 



curve, 



n(s) 



/ cos^(s)\ , , _ /— sin#(s 
v sin0(s)J' T[S) ~ \ cos9(s) 

We infer that the jacobian matrix of the change of variables is 



/ dx\ 


dx\ \ 


ds 


dr 


dx 2 


dX2 


\~ds~ 


dr / 



-(1 + k(s)t) sin 6(s) —cos9(s 
[1 + K(s)r) cos 9(s) — sin^(s^ 



and therefore the jacobian of the change of variables is |1 + k(s)t|. 
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